Motion vector coding apparatus, method and program for coding motion vector, motion vector decoding apparatus, and method and program for decoding motion vector

ABSTRACT

According to exemplary embodiments of the present invention, references to a motion vector in a neighboring block of an enhancement layer or in a block of a previously coded picture, and a motion vector of a base layer are equally handled. Accordingly, a motion_prediction_flag with identification information can be integrated for further improved coding efficiency.

TECHNICAL FIELD

The present invention relates to coding and decoding of an image, andmore particularly to coding and decoding of a motion vector for motioncompensation.

BACKGROUND ART

H.264/Moving Picture Experts Group (MPEG)-4 Advanced Video Coding (AVC)(hereinafter abbreviated as H.264), is known as a coding method formoving image compression recording (International Organization forStandardization (ISO)/International Electrotechnical Commission (IEC)14496-10:2010 Information technology—Coding of audio-visual objects—Part10: Advanced Video Coding). Similar to conventional coding methods,H.264 performs motion compensation by referring to other pictures forimproved coding efficiency. Motion vectors can be coded in units ofmacro-blocks (16 pixels*16 pixels) or blocks (eight pixels*eightpixels). To code a motion vector, a predicted motion vector iscalculated by using a median value of motion vectors in neighboringblocks (left, above, and upper right). An error between the predictedmotion vector and the motion vector is then coded.

H.264 is capable of hierarchical coding, which is discussed in the AnnexG Scalable Video Coding section of H.264. In the case of spatialscalability, a motion vector in a block of a base layer can be magnifiedto generate a predicted motion vector (hereinafter, referred to as aninterlayer predicted motion vector). A motion vector of the enhancementlayer may be coded by using the interlayer predicted motion vectorinstead of an ordinary median value of motion vectors. Morespecifically, the magnified motion vector can be used whenmotion_prediction_flag is TRUE.

International standardization activities for coding methods of evenhigher efficiency as a successor to H.264 have been recently started.The International Organization for Standardization and the InternationalElectrotechnical Commission (ISO/IEC) and the InternationalTelecommunication Union (ITU) Telecommunication Standardization Sector(ITU-T) established the Joint Collaborative Team on Video Coding(JCT-VC) for that purpose. JCT-VC promotes standardization of the HighEfficiency Video Coding scheme (hereinafter, abbreviated as HEVC). InHEVC, Advanced Motion Vector Prediction (AMVP) is discussed as a newmotion vector coding method (JCT-VC document JCTVC-A124_r2.doc, theInternet<http://wftp3.itu.int/av-arch/jctvc-site/2010_(—)04_A_Dresden/>). AMVPuses not only a median value of motion vectors in neighboring blocks asa reference motion vector, but also the motion vectors in theneighboring blocks themselves as reference motion vectors. Aside fromthe motion vectors in the neighboring blocks, a motion vector in a blockat the same position of the previous picture in coding order(hereinafter, referred to as a temporal direction predicted motionvector) is also included in predicted motion vectors. Motion vectorsincluding the same components among such predicted motion vectors areintegrated to reduce target motion vectors, from which a closest motionvector is selected. A code for identifying the selected motion vector(hereinafter, referred to as a predicted vector index code) and aprediction error resulting from the prediction are then coded forimproved coding efficiency.

When hierarchical coding is implemented in HEVC, whether to refer tomotion vectors in an enhancement layer or in a base layer is selectedlike H.264. If reference motion vectors may be selected like H.264, theabove-described method may be used. However, the above-described methodhas an issue of failing to improve the coding efficiency because both anAMVP-based predicted vector index code and a flag for selectingprediction of an interlayer motion vector are needed.

CITATION LISt Non Patent Literature

NPL 1: International Organization for Standardization(ISO)/International Electrotechnical Commission (IEC) 14496-10:2010Information technology—Coding of audio-visual objects—Part 10: AdvancedVideo Coding

NPL 2: JCT-VC document JCTVC-A124_r2.doc, the Internet<http://wftp3.itu.int/av-arch/jctvc-site/2010_(—)04_A_Dresden/>

SUMMARY OF INVENTION

The present invention is directed to coding a motion vector of anenhancement layer of hierarchical coding with a simpler configuration toimprove the coding efficiency.

According to an aspect of the present invention, a motion vector codingapparatus includes a motion vector calculation unit configured tocalculate a motion vector in a block of an enhancement layer to becoded, an interlayer predicted motion vector extraction unit configuredto extract a motion vector in a block at a position corresponding to theblock to be coded on image data of a base layer as an interlayerpredicted motion vector, a neighboring reference motion vectorextraction unit configured to extract a motion vector in a block of theenhancement layer neighboring the block to be coded as a neighboringreference motion vector, a selection unit configured to select apredicted motion vector for predicting a motion vector of theenhancement layer to be coded from a reference motion vector groupincluding the neighboring reference motion vector and the interlayerpredicted motion vector, a motion vector prediction unit configured topredict a motion vector to be coded using the selected predicted motionvector, and calculate a prediction error, and a coding unit configuredto code identification information for identifying the selectedpredicted motion vector and the prediction error.

According to exemplary embodiments of the present invention, referencesto a motion vector in a neighboring block of an enhancement layer or ina block of a previously coded picture, and a motion vector of a baselayer are equally handled. Accordingly, a motion_prediction_flag withidentification information can be integrated for further improved codingefficiency.

Further features and aspects of the present invention will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 is a block diagram illustrating a configuration of an imagecoding apparatus according to a first exemplary embodiment.

FIG. 2 is a detailed block diagram of a motion vector coding unit 102 inthe image coding apparatus according to the first exemplary embodiment.

FIG. 3 is a detailed block diagram of a motion vector coding unit 105 inthe image coding apparatus according to the first exemplary embodiment.

FIG. 4 is a flowchart illustrating processing for coding a motion vectorof a base layer by the image coding apparatus according to the firstexemplary embodiment.

FIG. 5 is a flowchart illustrating processing for coding a motion vectorof an enhancement layer by the image coding apparatus according to thefirst exemplary embodiment.

FIG. 6 is a diagram of an example of blocks and motion vectors.

FIG. 7 is a block diagram illustrating a configuration of an imagedecoding apparatus according to a second exemplary embodiment.

FIG. 8 is a detailed block diagram of a motion vector decoding unit 701according to the second exemplary embodiment.

FIG. 9 is a detailed block diagram of a motion vector decoding unit 704according to the second exemplary embodiment.

FIG. 10 is a flowchart illustrating processing for decoding a motionvector of a base layer by the image decoding apparatus according to thesecond exemplary embodiment.

FIG. 11 is a flowchart illustrating processing for decoding a motionvector of an enhancement layer by the image decoding apparatus accordingto the second exemplary embodiment.

FIG. 12 is a detailed block diagram of a motion vector coding unit 105in an image coding apparatus according to a third exemplary embodiment.

FIG. 13 is a flowchart illustrating processing for coding a motionvector of an enhancement layer by the image coding apparatus accordingto the third exemplary embodiment.

FIG. 14 is a detailed block diagram of a motion vector decoding unit 704according to a fourth exemplary embodiment.

FIG. 15 is a flowchart illustrating processing for decoding a motionvector of an enhancement layer by an image decoding apparatus accordingto the fourth exemplary embodiment.

FIG. 16 is a diagram of an example of blocks and motion vectors.

FIG. 17 is a block diagram illustrating an example of a hardwareconfiguration of a computer applicable to an image coding apparatus anda decoding apparatus of the exemplary embodiments of the presentinvention.

DESCRIPTION OF EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

A first exemplary embodiment of the present invention will be describedbelow with reference to the drawings. FIG. 1 is a block diagramillustrating an image coding apparatus according to the presentexemplary embodiment. In FIG. 1, image data is input to a terminal 106.A frame memory 107 stores the input image data picture by picture. Amagnification unit 113 magnifies the input image data by a magnificationn/m (n and m are positive numbers). A frame memory 114 stores themagnified image data picture by picture.

Prediction units 108 and 115 segment image data into a plurality ofblocks and perform intraframe prediction, such as intra prediction, andinter prediction by the motion compensation to generate predicted imagedata. The prediction methods are not limited thereto. The predictionunits 108 and 115 further calculate prediction errors from the inputimage data and the predicted image data, and output the calculatedprediction errors. For inter prediction, the prediction units 108 and115 refer to other coded pictures to calculate and output motionvectors. The prediction units 108 and 115 also output information neededfor prediction, such as information about an intra prediction mode,along with the prediction errors.

Transform and quantization units 109 and 116 orthogonally transform theprediction errors in units of blocks to obtain transform coefficients,and further perform quantization to obtain quantization coefficients.Frame memories 112 and 119 store reproduced image data.

Image reproduction units 111 and 118 input the quantization coefficientsin units of blocks, perform inverse quantization to obtain transformcoefficients, and further perform inverse orthogonal transform toreproduce prediction errors. In addition, the image reproduction units111 and 118 generate predicted image data from the motion vectors orprediction-related information from the prediction units 108 and 115 byreferring to the frame memories 112 and 119 when necessary. The imagereproduction units 111 and 118 finally generate reproduced image datafrom the resulting predicted image data and the reproduced predictionerrors, and output the reproduced image data. Coefficient coding units110 and 117 code the quantization coefficients output from the transformand quantization units 109 and 116 to generate quantization coefficientcode data.

A motion vector coding apparatus 100 according to the first exemplaryembodiment of the present invention includes motion vector storage units101 and 104 which store motion vectors in units of blocks. The motionvector storage unit 104 stores motion vectors generated by theprediction unit 108. The motion vector storage unit 101 stores motionvectors generated by the prediction unit 115. Motion vector coding units1102 and 105 code motion vectors in blocks generated by the predictionunits 108 and 115 to generate motion vector code data. A motion vectormagnification unit 103 magnifies a motion vector according to themagnification of the magnification unit 113. A magnification unit 120magnifies the image data from the frame memory 119 by a reciprocal ofthe magnification of the magnification unit 113, i.e., m/n.

Integration coding units 121 and 122 generate code data of headerinformation and prediction-related information. The integration codingunits 121 and 122 also integrate the motion vector code data generatedby the motion vector coding units 102 and 105 with the quantizationcoefficient code data generated by the coefficient coding units 110 and117.

The frame memories 114 and 119, the prediction unit 115, the transformand quantization unit 116, the coefficient coding unit 117, the imagereproduction unit 118, the motion vector storage unit 101, and themotion vector coding unit 102 code image data of a base layer. The framememories 107 and 112, the prediction unit 108, the transform andquantization unit 109, the coefficient coding unit 110, the imagereproduction unit 111, the motion vector storage unit 104, and themotion vector coding unit 105 code image data of an enhancement layer.

A multiplexing unit 123 multiplexes the code data of the base layer andthe code data of the enhancement layer to form a bit stream. A terminal124 outputs the bit stream generated by the multiplexing unit 123 tooutside.

An operation for coding an image performed by the above-described imagecoding apparatus will be described below. First, an operation for codingimage data of the base layer will be described.

A single frame of image data input from the terminal 106 is stored intothe frame memory 107. The magnification unit 113 magnifies the imagedata stored in the frame memory 107 by a predetermined magnificationn/m. In the case of spatial scalability, the value of n/m is less than1. The magnified image data is stored into the frame memory 114. Themagnification is input to the magnification unit 120 and the motionvector magnification unit 103.

The prediction unit 115 performs intra prediction or motion compensatedinter prediction. In the case of inter prediction, the prediction unit115 refers to a coded picture stored in the frame memory 119 and theblock data of the base layer stored in the frame memory 114 to calculatea motion vector. The calculated motion vector is input to the imagereproduction unit 118 and the motion vector storage unit 101. The motionvector storage unit 101 stores the motion vectors in units of blocks.

In addition, the prediction unit 115 generates and inputs a predictionerror to the transform and quantization unit 116. The transform andquantization unit 116 generates and inputs a quantization coefficient tothe coefficient coding unit 117. The coefficient coding unit 117performs entropy coding on the quantization coefficient to generatequantization coefficient code data. The entropy coding method is notlimited in particular, and Golomb coding, arithmetic coding, and Huffmancoding may be used. The quantization coefficient, the motion vector, andprediction-related information are input to the image reproduction unit118. The image generation unit 118 generates a reproduced image of ablock to be coded, and stores the reproduced image into the frame memory119.

The motion vector coding unit 102 codes the motion vector generated bythe prediction unit 115 to generate motion vector code data. FIG. 2illustrates a detailed block diagram of the motion vector coding unit102. In FIG. 2, motion vector from the motion vector storage unit 101 isinput to a terminal 201. A terminal 202 receives the motion vector inthe block to be coded input from the prediction unit 115. A motionvector extraction unit 203 extracts motion vectors neighboring the blockto be coded or a motion vector of a previously coded picture via theterminal 201.

A motion vector integration unit 204 integrates motion vectors includingthe same components to generate a reference motion vector group. In thereference motion vector group, the integrated motion vectors arearranged in a predetermined order. The arrangement order is not limitedin particular, and the integrated motion vectors may be arranged basedon the order of the magnitude of the components, the descending order ofthe probability of occurrence, or the positions of the extracted blocks.Any arrangement method may be used as long as the same method is used ona decoding side.

A predicted motion vector selection unit 205 receives the motion vectorin the block to be coded from the terminal 202, and the reference motionvector group from the motion vector integration unit 204 as inputs. Thepredicted motion vector selection unit 205 selects a motion vectorclosest to the motion vector in the block to be coded from the inputreference motion vector group as a predicted motion vector. Thepredicted motion vector selection unit 205 outputs an identificationnumber for identifying the selected predicted motion vector in thereference motion vector group, and the selected predicted motion vector.An index coding unit 206 codes the output identification number togenerate identification information code data. The method for coding theidentification number is not limited in particular, and Golomb coding,arithmetic coding, and Huffman coding may be used.

A prediction unit 207 outputs a prediction error that occurs when themotion vector in the block to be coded is predicted by the inputpredicted motion vector. A motion vector error coding unit 208 codes theprediction error of the motion vector to generate motion vectorprediction error code data. The method for coding the prediction errorof the motion vector is not limited in particular, and Golomb coding,arithmetic coding, and Huffman coding may be used. A code forming unit209 shapes the identification information code data and the motionvector prediction error code data into motion vector code data. Aterminal 201 outputs the generated motion vector code data to theintegration coding unit 122 in FIG. 1.

An operation for coding a motion vector of a base layer by theabove-described configuration will be described below.

The motion vector extraction unit 203 receives motion vectors inneighboring blocks of a block to be coded of the base layer and atemporal direction predicted motion vector via the terminal 201. Suchmotion vectors are input to the motion vector integration unit 204. Themotion vector integration unit 204 compares the components of the motionvectors and integrates the same ones to generate a reference motionvector group.

FIG. 6 illustrates how motion vectors are extracted and integrated. Ann-th picture 600 of the enhancement layer includes a block 603 to becoded. Blocks 604, 605, and 606 are adjacent to the block 603 to becoded. An (n-1)-th picture 601 of the enhancement layer includes a block607 that lies at the same position as the block 603. A motion vector 609is included in the block 603 to be coded. Motion vectors 610, 611, 612,and 613 are included in the respective blocks 604, 605, 606, and 607.Suppose that the motion vectors 611 and 612 have the same components. Insuch a case, the resulting reference motion vector group includes themotion vectors 613, 612, and 610.

The generated reference motion vector group is input to the predictedmotion vector selection unit 205. The predicted motion vector selectionunit 205 compares the motion vector in the block to be coded input fromthe terminal 202 and the motion vectors in the reference motion vectorgroup. The predicted motion vector selection unit 205 selects a motionvector including the closest components to those of the motion vector inthe block to be coded from the reference motion vector group as apredicted motion vector. The predicted motion vector selection unit 205generates an identification number for indicating order to identify theselected motion vector in the reference motion vector group. The indexcoding unit 206 codes the identification number to generateidentification information code data.

The predicted motion vector is input to the prediction unit 207. Theprediction unit 207 predicts the motion vector in the block to be coded,calculates a prediction error, and outputs the prediction error to themotion vector error coding unit 208. The motion vector error coding unit208 codes the prediction error to generate motion vector predictionerror code data. The code forming unit 209 shapes the identificationinformation code data and the motion vector prediction error code data,and outputs the resulting motion vector code data from the terminal 210.

Returning to FIG. 1, the generated motion vector code data and thequantization coefficient code data generated by the coefficient codingunit 117 are input to the integration coding unit 122. The integrationcoding unit 122 generates code data in units of blocks. The generatedcode data is input to the multiplexing unit 123.

Next, an operation for coding image data of the enhancement layer willbe described.

The magnification unit 120 magnifies image data of the base layer,stored in the frame memory 119, by m/n to generate reference image dataof the base layer. The prediction unit 108 divides the image data storedin the frame memory 107 into blocks, and performs intra prediction ormotion compensated inter prediction block by block. In the case of interprediction, the prediction unit 108 refers to a coded picture stored inthe frame memory 112 and the reference image data of base layer tocalculate a motion vector. The calculated motion vector is input to theimage reproduction unit 111 and the motion vector storage unit 104. Themotion vector storage unit 104 stores the motion vector in units ofblocks. In addition, the prediction unit 108 generates and inputs aprediction error to the transform and quantization unit 109. Thetransform and quantization unit 109 generates and inputs a quantizationcoefficient to the coefficient coding unit 110. Similar to thecoefficient coding unit 117, the coefficient coding unit 110 performsentropy coding on the quantization coefficient to generate quantizationcoefficient code data. The quantization coefficient, the motion vector,and prediction-related information are input to the image reproductionunit 111. Similar to the image reproduction unit 118, the imagereproduction unit 111 generates a reproduced image of a block to becoded, and stores the reproduced image into the frame memory 112.

The motion vector coding unit 105 codes the motion vector generated bythe prediction unit 108 to generate motion vector code data. Prior tothe coding of a motion vector in the motion vector coding unit 105, themotion vector magnification unit 103 extracts a motion vector at theposition of a block of the base layer corresponding to the block to becoded of the enhancement layer from the motion vector storage unit 101.The motion vector magnification unit 103 magnifies the extracted motionvector by m/n according to the magnification (n/m) output from themagnification unit 113, and outputs the resulting interlayer predictedmotion vector to the motion vector coding unit 105.

FIG. 6 illustrates how motion vectors are extracted and integrated. Ann-th picture 602 of the base layer corresponding to the n-th picture 600of the enhancement layer. A block 608 lies in a position correspondingto the block 603 to be coded of the enhancement layer. The block 608 ofthe base layer includes a motion vector 614. The motion vector 614 ismagnified into an interlayer predicted motion vector which has the sameprecision as the resolution of the enhancement layer. Suppose that themotion vectors 611 and 612 and the interlayer predicted motion vectorhave the same components. In such a case, the resulting reference motionvector group includes the interlayer predicted motion vector and themotion vectors 613 and 610.

FIG. 3 illustrates a detailed block diagram of the motion vector codingunit 105. In

FIG. 3, blocks implementing similar functions to those of the blocks inFIG. 2 are designated by the same reference numerals. Descriptionthereof will be omitted. A terminal 301 inputs an interlayer predictedmotion vector from the motion vector magnification unit 103. A motionvector extraction unit 303 extracts motion vectors neighboring the blockto be coded of the enhancement layer or a motion vector of a previouslycoded picture via the terminal 201, and adds the interlayer predictedmotion vector from the terminal 301. A motion vector integration unit304 integrates motion vectors including the same components to generatea reference motion vector group. In the reference motion vector group,the integrated motion vectors are arranged in a predetermined order. Thearrangement order is not limited in particular, whereas the interlayerpredicted motion vector is arranged at the top. The rest of the motionvectors may be arranged based on the order of the magnitude of thecomponents, the descending order of the probability of occurrence, orthe positions of the extracted blocks. The terminal 201 is connected tothe motion vector storage unit 104. The terminal 202 is connected to theprediction unit 108. The terminal 210 is connected to the integrationcoding unit 121.

An operation for coding a motion vector of an enhancement layer by theabove-described configuration will be described below.

The motion vector extraction unit 303 inputs motion vectors inneighboring blocks of the block of the base layer to be coded, atemporal direction predicted motion vector, and an interlayer predictedmotion vector. The motion vector integration unit 304 compares thecomponents of such motion vectors and integrates the same ones togenerate a reference motion vector group. In the reference motion vectorgroup, the integrated motion vectors are arranged in a predeterminedorder. The generated reference motion vector group is input to thepredicted motion vector selection unit 205. Sub-sequently, similar tothe motion vector coding unit 102 of the base layer, the motion vectorin the block to be coded is compared with the motion vectors in thereference motion vector group to select a predicted motion vector. Anidentification number of the selected predicted motion vector isgenerated and coded to generate identification information code data. Aprediction error of the motion vector in the block to be coded iscalculated and coded to generate motion vector prediction error codedata. The code forming unit 209 shapes the identification informationcode data and the motion vector prediction error code data, and outputsthe resulting motion vector code data from the terminal 210.

Returning to FIG. 1, the generated motion vector code data and thequantization coefficient code data generated by the coefficient codingunit 110 are input to the integration coding unit 121. The integrationcoding unit 121 generates code data in units of blocks. The generatedcode data is input to the multiplexing unit 123.

The multiplexing unit 123 multiplexes such code data according to apredetermined format, and outputs the resulting bit stream from theterminal 124 to outside.

FIG. 4 is a flowchart illustrating processing for coding a motion vectorof a base layer by the image coding apparatus according to the firstexemplary embodiment. In step S401, the image coding apparatus inputs amotion vector in a block to be coded of the base layer, and stores themotion vector for reference in subsequent stages. In step S402, theimage coding apparatus extracts motion vectors in neighboring blocks ofthe block to be coded of the base layer or in a block at the sameposition of a coded picture of the base layer from stored coded motionvectors of the base layer.

In step S403, the image coding apparatus integrates motion vectorsincluding the same components among the motion vectors extracted in stepS402. The image coding apparatus rearranges the resultant in apredetermined order to generate a reference motion vector group. In stepS404, the image coding apparatus compares the reference motion vectorgroup generated in step S403 with the motion vector in the block to becoded of the base layer to select the closest motion vector as apredicted motion vector of the base layer, and generates identificationinformation.

In step S405, the image coding apparatus calculates a prediction errorof the motion vector to be coded using the predicted motion vector. Instep S406, the image coding apparatus codes the identificationinformation generated in step S404. In step S407, the image codingapparatus codes the prediction error of the motion vector of the baselayer.

FIG. 5 is a flowchart illustrating processing for coding a motion vectorof an enhancement layer by the image coding apparatus according to thefirst exemplary embodiment. In FIG. 5, steps for implementing similarfunctions to those of the steps in FIG. 4 are designated by the samereference numerals. Description thereof will be omitted. In step S501,the image coding apparatus inputs a motion vector in a block to be codedof the enhancement layer, and stores the motion vector for reference insubsequent stages.

In step S502, the image coding apparatus extracts motion vectors inneighboring blocks of the block to be coded of the enhancement layer orin a block at the same position of a coded picture of the enhancementlayer from stored motion vectors of the enhancement layer. In step S503,the image coding apparatus extracts a motion vector in a block of thebase layer corresponding to the position of the block to be coded of theenhancement layer.

In step S504, the image coding apparatus magnifies the extracted motionvector of the base layer to calculate an interlayer predicted motionvector. In step S505, the image coding apparatus integrates motionvectors including the same components among the motion vectors extractedin step S502 and the interlayer predicted motion vector. The imagecoding apparatus rearranges the resultant in a predetermined order togenerate a reference motion vector group. In subsequent steps S404 toS407, similar to the coding of a motion vector of the base layer in FIG.4, the image coding apparatus determines a predicted motion vector andgenerates identification information. The image coding apparatus furthercalculates a prediction error from the predicted motion vector. Theimage coding apparatus codes the generated identification informationand the prediction error of the motion vector.

According to the above-described configuration and operation, the imagecoding apparatus can efficiently code the motion vector of theenhancement layer by using the motion vector of the base layer. In thecase where the magnification factor is 1, the magnification units 113and 120 and the motion vector magnification unit 103 may be omitted.

The motion vector integration unit 204 may always allocate an interlayerpredicted motion vector to the top of a reference motion vector group.Accordingly, chances for a shorter code to be assigned to an interlayerpredicted motion vector which is highly likely to be referred to can beincreased, and the coding efficiency can be further improved.

The motion vector extraction unit 203 inputs a motion vector in a blockat the same position of a picture in which a temporal directionpredicted motion vector is coded. However, this is not restrictive. Forexample, the input of a temporal direction predicted motion vector maybe omitted if there is an interlayer predicted motion vector.

The motion vector integration unit 304 is described to arrange aninterlayer predicted motion vector at the top. However, the motionvector integration unit 304 may arrange an interlayer predicted motionvector after a temporal direction predicted motion vector. In addition,similar to the motion vector integration unit 204, the arrangement neednot be particularly fixed.

According to the present exemplary embodiment, the motion vectorintegration units 204 and 304 integrate motion vectors including thesame components in order to improve the coding efficiency. However, themotion vector integration units 204 and 304 are not limited to the aboveconfiguration. The motion vector integration units 204 and 304 mayselect a fixed number of motion vectors. The motion vector integrationunits 204 and 304 may even be omitted.

A base layer coding apparatus and an enhancement layer coding apparatusmay be provided separately, and the magnification units 113 and 120 maybe provided outside of the image coding apparatus.

The blocks lying in the positions illustrated in FIG. 6 are referred tofor the motion vectors of the reference motion vector group. However,the blocks and the motion vectors are not limited to this example. Forexample, a motion vector in a block adjacent to the right of the block604 may be added. A motion vector corresponding to a median value ofsuch motion vectors may be added. The motion vector to be referred to inthe base layer is not limited to that of the same position, either. Asillustrated in FIG. 16, a motion vector 1604 in a block 1601 below theblock 608, a motion vector 1605 in a block 1602 on the lower right, anda motion vector 1606 in a block 1603 on the right may be referred to.Motion vectors in other blocks may also be referred to.

FIG. 7 is a block diagram illustrating a configuration of an imagedecoding apparatus according to a second exemplary embodiment of thepresent invention. The present exemplary embodiment will be describedusing the decoding of code data generated in the first exemplaryembodiment as an example.

A terminal 706 inputs a coded bit stream. A separation unit 707separates the bit stream into code data of the base layer and code dataof the enhancement layer. The separation unit 707 performs an operationreverse to that of the multiplexing unit 123 in FIG. 1. Integrationdecoding units 708 and 715 decode code data on header information andprediction-related information about the base layer and the enhancementlayer. The integration decoding units 708 and 715 separate quantizationcoefficient code data and motion vector code data, and output theresultant to subsequent stages. Coefficient decoding units 709 and 716perform an operation reverse to that of the coefficient coding units 117and 110 in FIG. 1 to reproduce quantization coefficients. Inversequantization and inverse transform units 710 and 717 perform anoperation reverse to that of the transform and quantization units 116and 109 in FIG. 1. The inverse quantization and inverse transform units710 and 717 reproduce orthogonal transform coefficients by inversequantization, and generate prediction errors by inverse orthogonaltransform.

A motion vector decoding apparatus 700 according to the presentexemplary embodiment of the present invention includes vector decodingunits 701 and 704. The motion vector decoding units 701 and 704 decodemotion vector code data to reproduce motion vectors in blocks to bedecoded. Motion vector storage units 702 and 705 store the motionvectors in units of blocks. A motion vector magnification unit 703magnifies a motion vector according to the same magnification as that ofthe magnification unit 113 in FIG. 1. The method for calculating themagnification is not limited in particular. The ratio between theresolution of the base layer and that of the enhancement layer may becalculated. A predetermined value may be set.

Frame memories 712, 713, 719, and 720 store image data of reproducedpictures.

Image reproduction units 711 and 718 input the motion vectors, refer toimage data on decoded pictures, perform motion compensation, andreproduce image data using the decoded prediction errors. Amagnification unit 714 magnifies the image data of the base layeraccording to the same magnification as that of the magnification unit113 in FIG. 1. The method for calculating the magnification is notlimited in particular. The ratio between the resolution of the baselayer and that of the enhancement layer may be calculated. Apredetermined value may be set. A terminal 721 outputs the image data ofthe base layer. A terminal 722 outputs the image data of the enhancementlayer.

The integration decoding unit 708, the coefficient decoding unit 708,the inverse quantization and inverse transform unit 710, the motionvector decoding unit 701, the motion vector storage unit 702, the imagereproduction unit 711, and the frame memories 712 and 713 decode thecode data of the base layer and reproduce the image data of the baselayer. The integration decoding unit 715, the coefficient decoding unit716, the inverse quantization and inverse transform unit 717, the motionvector decoding unit 704, the motion vector storage unit 705, the imagereproduction unit 718, and the frame memories 719 and 720 decode thecode data of the enhancement layer and reproduce the image data of theenhancement layer.

An operation for decoding an image by the above-described image decodingapparatus will be described below. In the present exemplary embodiment,the image decoding apparatus decodes a bit stream generated in the firstexemplary embodiment.

In FIG. 7, the bit stream input from the terminal 706 is input to theseparation unit 707. The separation unit 707 separates the bit streaminto code data of the base layer and code data of the enhancement layer.The former is input to the integration decoding unit 708. The latter isinput to the integration decoding unit 715.

First, an operation for decoding the code data of the base layer will bedescribed. The code data of the base layer is input to the integrationdecoding unit 708. The integration decoding unit 708 decodes the codedata on header information and prediction-related information, separatesquantization coefficient code data and motion vector code data, inputsthe motion vector code data to the motion vector decoding unit 701, andinputs the quantization coefficient code data to the coefficientdecoding unit 709.

The coefficient decoding unit 709 decodes the quantization coefficientcode data to reproduce a quantization coefficient. The reproducedquantization coefficient is input to the inverse quantization andinverse transform unit 710. The inverse quantization and inversetransform unit 710 performs inverse quantization on the reproducedquantization coefficient to generate an orthogonal transformcoefficient, and further applies inverse orthogonal transform thereto toreproduce a prediction error. The reproduced prediction error is inputto the image reproduction unit 711.

The motion vector decoding unit 701 decodes the motion vector code datato reproduce a motion vector in a block to be decoded of the base layer.FIG. 8 illustrates a detailed block diagram of the motion vectordecoding unit 701. In FIG. 8, a terminal 801 inputs the motion vectorcode data of the block to be decoded from the integration decoding unit708. A terminal 802 inputs a motion vector from the motion vectorstorage unit 702. A motion vector extraction unit 803 extracts motionvectors neighboring the block to be decoded of the base layer or amotion vector of a previously decoded picture via the terminal 802.

A motion vector integration unit 804 performs a similar operation tothat of the motion vector integration unit 204 in FIG. 2. That is, themotion vector integration unit 804 integrates motion vectors includingthe same components to generate a reference motion vector group. A codeseparation unit 805 separates the motion vector code data intoidentification information code data and motion vector prediction errorcode data. An index decoding unit 806 decodes the identificationinformation code data to reproduce identification information. The indexdecoding unit 806 performs an operation reverse to that of the indexcoding unit 206 in FIG. 2 for decoding.

A motion vector error decoding unit 807 decodes the motion vectorprediction error code data to reproduce the prediction error of themotion vector in the block to be decoded. The motion vector errordecoding unit 807 performs an operation reverse to that of the motionvector error coding unit 208 in FIG. 2 for decoding. A predicted motionvector selection unit 808 selects a predicted motion vector from thereference motion vector group using the reproduced identificationinformation. A motion vector reproduction unit 809 reproduces the motionvector in the block to be decoded of the base layer from the selectedpredicted motion vector and the reproduced prediction error of themotion vector. A terminal 810 outputs the reproduced motion vector tothe image reproduction unit 711 and the motion vector storage unit 702in FIG. 7.

An operation for decoding a motion vector of the base layer by theabove-described configuration will be described below. The motion vectorcode data of the block to be decoded of the base layer input from theterminal 801 is input to the code separation unit 805. The codeseparation unit 805 separates the motion vector code data intoidentification information code data and motion vector prediction errorcode data. The former is input to the index decoding unit 806. Thelatter is input to the motion vector error decoding unit 807.

The index decoding unit 806 decodes the identification information codedata to reproduce identification information. The motion vector errordecoding unit 807 decodes the motion vector prediction error code datato reproduce the prediction error of the motion vector in the block tobe decoded of the base layer. As illustrated in FIG. 6, the motionvector extraction unit 803 extracts the motion vectors in theneighboring blocks of the block of the base layer to be decoded and thetemporal direction predicted motion vector from the motion vectorstorage unit 702 via the terminal 802. The extracted motion vectors areinput to the motion vector integration unit 804. The motion vectorintegration unit 804 generates a reference motion vector group similarto the motion vector integration unit 204 in FIG. 2.

The predicted motion vector selection unit 808 inputs the identificationinformation from the index decoding unit 806 and selects a predictedmotion vector from the reference motion vector group. The selectedpredicted motion vector is input to the motion vector generation unit809 together with the reproduced prediction error of the motion vectorin the block to be decoded of the base layer. The motion vectorreproduction unit 809 reproduces the motion vector in the block to bedecoded from the reproduced prediction error and the predicted motionvector. The reproduced motion vector is output via the terminal 810.

Returning to FIG. 7, the reproduced motion vector is input to the imagereproduction unit 711 and the motion vector storage unit 702. The motionvector storage unit 702 stores motion vectors in units of blocks. Theimage reproduction unit 711 calculates predicted image data of the blockto be decoded of the base layer using the reproduced motion vector andbased on the frame memory 713. The image reproduction unit 711 furtheris input the prediction error reproduced by the inverse quantization andinverse transform unit 710, and reproduces image data of the base layerusing the predicted image data. The reproduced image data of the baselayer is stored into the frame memories 712 and 713 and used forreference. The reproduced image data of the base layer is output fromthe terminal 721.

Next, an operation for decoding code data of the enhancement layer willbe described. Prior to the decoding of a motion vector by the motionvector decoding unit 704, the motion vector magnification unit 703extracts a motion vector at the position of a block of the base layercorresponding to the block to be decoded of the enhancement layer fromthe motion vector storage unit 702. The motion vector magnification unit703 magnifies the extracted motion vector by m/n according to themagnification (n/m), and outputs the resulting interlayer predictedmotion vector to the motion vector decoding unit 704.

The code data of the enhancement layer is input to the integrationdecoding unit 715. The integration decoding unit 715 decodes the codedata on header information and prediction-related information, separatesquantization coefficient code data and motion vector code data, inputsthe motion vector code data to the motion vector decoding unit 704, andinputs the quantization coefficient code data to the coefficientdecoding unit 716.

The coefficient decoding unit 716 decodes the quantization coefficientcode data to reproduce a quantization coefficient. The reproducedquantization coefficient is input to the inverse quantization andinverse transform unit 717. The inverse quantization and inversetransform unit 717 performs inverse quantization on the reproducedquantization coefficient to generate an orthogonal transformcoefficient, and further applies inverse orthogonal transform thereto toreproduce a prediction error. The reproduced prediction error is inputto the image reproduction unit 718.

The motion vector decoding unit 704 decodes the motion vector code datato reproduce the motion vector in the block to be decoded of theenhancement layer. FIG. 9 illustrates a detailed block diagram of themotion vector decoding unit 704. In FIG. 9, blocks implementing similarfunctions to those of the blocks in FIG. 8 are designated by the samereference numerals. Description thereof will be omitted. In FIG. 9, aterminal 901 inputs a motion vector from the motion vector magnificationunit 703. A motion vector extraction unit 903 is input motion vectorsneighboring the block to be decoded of the enhancement layer or a motionvector of a previously decoded picture as illustrated in FIG. 6 via theterminal 901. The motion vector extraction unit 903 is also input theinterlayer predicted motion vector input from the terminal 901. A motionvector integration unit 904 performs a similar operation to that of themotion vector integration unit 804 in FIG. 8. That is, the motion vectorintegration unit 904 integrates motion vectors including the samecomponents to generate a reference motion vector group.

In the reference motion vector group, the integrated motion vectors arearranged in a predetermined order. The arrangement order is not limitedin particular, whereas the interlayer predicted motion vector isarranged at the top. The rest of the motion vectors may be arrangedbased on the order of the magnitude of the components, the descendingorder of the probability of occurrence, or the positions of theextracted blocks. A terminal 801 is connected to the integrationdecoding unit 715. A terminal 802 is connected to the motion vectorstorage unit 705. A terminal 810 is connected to the motion vectorstorage unit 705 and the image reproduction unit 718.

An operation for decoding a motion vector of the enhancement layer bythe above-described configuration will be described. The motion vectorextraction unit 903 is input motion vectors in neighboring blocks of theblock to be decoded of the enhancement layer and a temporal directionpredicted motion vector from the motion vector storage unit 705 via theterminal 802. The motion vector extraction unit 903 is further input aninterlayer predicted motion vector from the terminal 901. Such motionvectors are input to the motion vector integration unit 904. The motionvector integration unit 904 generates a reference motion vector groupsimilar to the motion vector integration unit 804 in FIG. 8.Subsequently, similar to the motion vector decoding unit 701 of the baselayer, the motion vector decoding unit 704 selects a predicted motionvector from the reference motion vector group based on theidentification information. The motion vector decoding unit 704reproduces the motion vector in the block to be decoded of theenhancement layer using the selected predicted motion vector and thereproduced prediction error of the motion vector in the block to bedecoded of the enhancement layer. The reproduced motion vector is outputvia the terminal 810.

Returning to FIG. 7, the reproduced motion vector is input to the imagereproduction unit 718 and the motion vector storage unit 705. The motionvector storage unit 705 stores motion vectors in units of blocks. Theimage reproduction unit 718 is input the image data magnified by themagnification unit 714 by reading corresponding image data from theframe memory 713 or the image data by reading the frame memory 713 usingthe reproduced motion vector. The image reproduction unit 718 calculatespredicted image data of the block to be decoded of the enhancement layerfrom the image data. The image reproduction unit 718 is further inputsthe prediction error reproduced by the inverse quantization and inversetransform unit 717, and reproduces image data of the enhancement layerusing the predicted image data. The reproduced image data of theenhancement layer is stored in the frame memories 719 and 720 and usedfor reference. The reproduced image data of the enhancement layer isoutput from the terminal 722.

FIG. 10 is a flowchart illustrating processing for decoding a motionvector of the base layer by the image decoding apparatus according tothe second exemplary embodiment. step S1001, the image decodingapparatus extracts motion vectors in neighboring blocks of the block tobe decoded of the base layer and a temporal direction predicted motionvector from stored motion vectors. In step S1002, the image decodingapparatus integrates motion vectors including the same components amongthe motion vectors extracted in step S1001, and rearranges the resultantin a predetermined order to generate a reference motion vector group.

In step S1003, the image decoding apparatus decodes identificationinformation code data to reproduce identification information. In stepS1004, the image decoding apparatus selects a predicted motion vectorfrom the reference motion vector group generated in step S1002 based onthe identification information reproduced in step S1003. In step S1005,the image decoding apparatus decodes motion vector prediction error codedata to reproduce the prediction error of the motion vector in the blockto be decoded of the base layer.

In step S1006, the image decoding apparatus reproduces the motion vectorin the block to be decoded of the base layer from the predicted motionvector selected in step S1004 and the prediction error of the motionvector reproduced in step S1005. In step S1007, the image decodingapparatus outputs the reproduced motion vector, and stores the motionvector for reference in subsequent stages.

FIG. 11 is a flowchart illustrating processing for decoding a motionvector of the enhancement layer by the image decoding apparatusaccording to the second exemplary embodiment. In FIG. 11, steps forimplementing similar functions to those of the steps in FIG. 10 aredesignated by the same reference numerals. Description thereof will beomitted.

In step S1101, the image decoding apparatus extracts motion vectors inneighboring blocks of the block to be decoded of the enhancement layeror a temporal direction predicted motion vector of the enhancement layerfrom stored decoded motion vectors of the enhancement layer. In stepS1102, the image decoding apparatus extracts a motion vector in a blockof the base layer corresponding to the position of the block to bedecoded of the enhancement layer. In step S1103, the image decodingapparatus magnifies the extracted motion vector of the base layer tocalculate an interlayer predicted motion vector. In step S1104, theimage decoding apparatus integrates motion vectors including the samecomponents among the motion vectors extracted in step S1102 and theinterlayer predicted motion vector, and rearranges the resultant in apredetermined order to generate a reference motion vector group.

In step S1105, the image decoding apparatus decodes identificationinformation code data to reproduce identification information. In stepS1106, the image decoding apparatus selects a predicted motion vectorfrom the reference motion vector group generated in step S1104 based onthe reproduced identification information. Subsequently, similar to thedecoding of the motion vector of the base layer in FIG. 10, the imagedecoding apparatus decodes motion vector prediction error code data toreproduce the prediction error of the motion vector. The image decodingapparatus reproduces the motion vector in the block to be decoded of theenhancement layer from the selected predicted motion vector and theprediction error of the motion vector. The image decoding apparatusoutputs and stores the reproduced motion vector.

According to the above-described configuration and operation, a bitstream which is generated in the first exemplary embodiment andefficiently coded using the motion vectors of the base layer can bedecoded to obtain a reproduced image. In the case where themagnification factor is 1, the magnification unit 714 and the motionvector magnification unit 703 may be omitted.

The motion vector integration unit 904 may always allocate an interlayerpredicted motion vector to the top of a reference motion vector group.Accordingly, chances for a shorter code to be assigned to an interlayerpredicted motion vector which is highly likely to be referred to can beincreased, and the decoding efficiency can be further improved.

The motion vector extraction unit 903 inputs a temporal directionpredicted motion vector. However, this is not restrictive. For example,the input of a temporal direction predicted motion vector may be omittedif there is an interlayer predicted motion vector.

The motion vector integration unit 904 is described to arrange aninterlayer predicted motion vector at the top. However, the motionvector integration unit 904 may arrange the interlayer predicted motionvector after a temporal direction predicted motion vector. In addition,similar to the motion vector integration unit 204, the arrangement neednot be particularly fixed.

In the present exemplary embodiment, the magnification is input from themotion vector magnification unit 703. However, the input of themagnification is not limited to this example. The ratio between theresolution of the base layer and that of the enhancement layer may beindependently calculated. A predetermined value may be set.

According to the present exemplary embodiment, the motion vectorintegration units 804 and 904 integrate motion vectors including thesame components in order to improve the coding efficiency of coded data.However, the motion vector integration units 804 and 904 are not limitedto this configuration. The motion vector integration units 804 and 904may select a fixed number of motion vectors. The motion vectorintegration units 804 and 904 may even be omitted.

A base layer decoding apparatus and an enhancement layer decodingapparatus may be provided separately, and the magnification unit 714 maybe provided outside of the image decoding apparatus.

The blocks lying in the positions illustrated in FIG. 6 are referred tofor the motion vectors of the reference motion vector group. However,the blocks and the motion vectors are not limited to this example. Forexample, a motion vector in a block adjacent to the right of the block604 may be added. A motion vector corresponding to a median value ofsuch motion vectors may be added. The motion vector to be referred to inthe base layer is not limited to that of the same position, either. Asillustrated in FIG. 16, a motion vector 1604 in a block 1601 below theblock 608, a motion vector 1605 in a block 1602 on the lower right, anda motion vector 1606 in a block 1603 on the right may be referred to.Motion vectors in other blocks may also be referred to.

An image coding apparatus according to a third exemplary embodimentincludes a similar configuration to that in FIG. 1 according to thefirst exemplary embodiment. A difference lies in the configuration ofthe motion vector coding unit 105 of the enhancement layer. Thus, thecoding of the base layer and the coding of the quantization coefficientof the enhancement layer are the same as in the first exemplaryembodiment, and description thereof will be omitted.

FIG. 12 is a block diagram illustrating a detailed configuration of themotion vector coding unit 105 according to the present exemplaryembodiment. In FIG. 12, parts implementing similar functions to those inFIG. 3 according to the first exemplary embodiment are designated by thesame reference numerals. Description thereof will be omitted. Similar tothe terminal 301 in FIG. 3, a terminal 1201 inputs an interlayerpredicted motion vector from the motion vector magnification unit 103. Amotion vector extraction unit 1203 extracts motion vectors neighboringthe block to be coded of the enhancement layer or a motion vector of apreviously coded picture which are input from the terminal 201. Apredicted motion vector selection unit 1205 is input a motion vector inthe block to be coded of the enhancement layer from the terminal 202, areference motion vector group from the motion vector integration unit304, and the interlayer predicted motion vector from the terminal 1201.The predicted motion vector selection unit 1205 selects a motion vectorclosest to the motion vector in the block to be coded from the inputreference motion vector group as a predicted motion vector.

An operation for coding a motion vector of the enhancement layer by theabove-described configuration will be described.

The motion vector extraction unit 1203 extracts the motion vectorsneighboring the block to be coded of the enhancement layer or the motionvector of the previously coded picture illustrated in FIG. 6. Theextracted motion vector is input to the motion vector integration unit304. Similar to the first exemplary embodiment, the motion vectorintegration unit 304 integrates motion vectors including the samecomponents among the input motion vectors to generate a reference motionvector group. The predicted motion vector selection unit 1205 selects amotion vector most similar to the motion vector in the block to be codedfrom the input reference motion vector group and the interlayerpredicted motion vector as a predicted motion vector. The predictedmotion vector selection unit 1205 outputs identification information foridentifying the selected predicted motion vector, and the predictedmotion vector.

A difference from the first exemplary embodiment lies in that theinterlayer predicted motion vector is handled as a selection candidateindependent of the reference motion vector group. The reference motionvector group may thus include a motion vector including the samecomponents as those of the interlayer predicted motion vector.Therefore, the predicted motion vector selection unit 1205 is configuredto select the interlayer predicted motion vector as a predicted motionvector if both the interlayer predicted motion vector and the motionvector in the reference motion vector group can be selected.

Subsequently, similar to the first exemplary embodiment, the indexcoding unit 206 codes identification information. The prediction unit207 calculates a prediction error from the motion vector in the block tobe coded and the predicted motion vector. The motion vector error codingunit 208 codes the prediction error.

FIG. 13 is a flowchart illustrating processing for coding a motionvector of the enhancement layer by the image coding apparatus accordingto the third exemplary embodiment. In FIG. 13, steps for implementingsimilar functions to those of the steps in FIG. 5 are designated by thesame reference numerals. Description thereof will be omitted.

Processing in steps S501 to S504 are similar to those of the firstexemplary embodiment. More specifically, the image coding apparatusextracts motion vectors neighboring the block to be coded of theenhancement layer or a moving vector in a block at the same position ofa coded picture of the enhancement layer from stored motion vectors ofthe enhancement layer, and calculates an interlayer predicted motionvector. In step S1301, the image coding apparatus integrates motionvectors including the same components among the motion vectorsneighboring the block to be coded of the enhancement layer or the motionvector of the previously coded picture, and rearranges the resultant ina predetermined order to generate a reference motion vector group. Instep S1302, the image coding apparatus adds the interlayer predictedmotion vector to the top of the generated reference motion vector group.

In step S404, the image coding apparatus selects a predicted motionvector from the reference motion vector group similar to the firstexemplary embodiment. In step S1303, if the predicted motion vectorincludes the same components as those of the interlayer predicted motionvector (YES in step S1303), the processing proceeds to step S1304.Whereas if not (NO in step S1303), the processing proceeds to step 5405.In step S1304, the image coding apparatus selects the interlayerpredicted motion vector as a predicted motion vector. Accordingly, theother motion vector including the same components may be prevented frombeing selected. In steps 5405 to 5407, similar to the processing forcoding the motion vector of the enhancement layer in FIG. 5 according tothe first exemplary embodiment, the image coding apparatus calculatesthe prediction error of the motion vector, codes the identificationinformation, and codes the prediction error of the motion vector.

According to the above-described configuration and operation, the imagecoding apparatus can efficiently code the motion vector of theenhancement layer by preferentially using the motion vector of the baselayer over other motion vectors. This is because the interlayerpredicted motion vector has extremely highly correlation with the motionvector in the block of the enhancement layer.

The blocks lying in the positions illustrated in FIG. 6 are referred tofor the motion vectors of the reference motion vector group. However,the blocks and the motion vectors are not limited to this example. Forexample, a motion vector in a block adjacent to the right of the block604 may be added. A motion vector corresponding to a median value ofsuch motion vectors may be added.

According to the present exemplary embodiment, the motion vectorintegration units 204 and 304 integrate motion vectors including thesame components in order to improve the coding efficiency. However, themotion vector integration units 204 and 304 are not limited to the aboveconfiguration. The motion vector integration units 204 and 304 mayselect a fixed number of motion vectors. The motion vector integrationunits 204 and 304 may even be omitted.

An image decoding apparatus according to a fourth exemplary embodimentincludes a similar configuration to that in FIG. 7 according to thesecond exemplary embodiment. A difference lies in the configuration ofthe motion vector decoding unit 704 of the enhancement layer. Thus, thedecoding of the base layer and the decoding of the quantizationcoefficient of the enhancement layer are the same as in the secondexemplary embodiment, and description thereof will be omitted. Thepresent exemplary embodiment will be described by using the decoding ofcode data generated in the third exemplary embodiment as an example.

FIG. 14 is a block diagram illustrating a detailed configuration of themotion vector decoding unit 704 according to the present exemplaryembodiment. In FIG. 14, parts providing similar functions to those inFIG. 9 according to the second exemplary embodiment are designated bythe same reference numerals. Description thereof will be omitted.Similar to the terminal 901 in FIG. 9, a terminal 1401 inputs aninterlayer predicted motion vector from the motion vector magnificationunit 703. Similar to the motion vector extraction unit 1203 of the thirdexemplary embodiment, a motion vector extraction unit 1403 extractsmotion vectors neighboring the block to be decoded of the enhancementlayer or a motion vector of a previously decoded picture. A predictedmotion vector selection unit 1405 selects a predicted motion vector froman interlayer predicted motion vector and a reference motion vectorgroup according to identification information.

An operation for decoding a motion vector of the enhancement layer bythe above-described configuration will be described.

The motion vector extraction unit 1403 extracts the motion vectorsneighboring the block to be decoded of the enhancement layer or themotion vector of the previously decoded picture illustrated in FIG. 6.The extracted motion vector is input to the motion vector integrationunit 804. Similar to the second exemplary embodiment, the motion vectorintegration unit 804 integrates motion vectors including the samecomponents among the input motion vectors to generate a reference motionvector group. Similar to the second exemplary embodiment, the indexdecoding unit 806 decodes identification information code data toreproduce identification information. The predicted motion vectorselection unit 1405 selects a predicted motion vector from an interlayerpredicted motion vector and the reference motion vector group accordingto the identification information. Subsequently, similar to the secondexemplary embodiment, the image decoding apparatus reproduces the motionvector to be decoded of the enhancement layer from the prediction errorof the motion vector and the selected predicted motion vector.

FIG. 15 is a flowchart illustrating processing for decoding a motionvector of the enhancement layer by the image decoding apparatusaccording to the fourth exemplary embodiment. In FIG. 15, steps forimplementing similar functions to those of the steps in FIG. 11 aredesignated by the same reference numerals. Description thereof will beomitted.

Processing in steps S1101 to S1103 are similar to those of the secondexemplary embodiment. More specifically, the image decoding apparatusextracts motion vectors neighboring the block to be decoded of theenhancement layer or a motion vector of a previously decoded picture,and calculates an interlayer predicted motion vector.

In step S1501, the image decoding apparatus integrates motion vectorsincluding the same components among the motion vectors neighboring theblock to be decoded of the enhancement layer or the motion vector of thepreviously decoded picture, and rearranges the resultant to generate areference motion vector group. In step S1502, the image decodingapparatus adds the interlayer predicted motion vector to the top of thegenerated reference motion vector group.

Subsequently, similar to the processing for decoding a motion vector ofthe enhancement layer in FIG. 11 according to the second exemplaryembodiment, the image decoding apparatus performs the processing insteps S1105 to S1007. More specifically, the image decoding apparatusdecodes identification information, selects a predicted motion vector,reproduces the prediction error of the motion vector, and reproduces themotion vector in the block to be decoded of the enhancement layer.

According to the above-described configuration and operation, the imagedecoding apparatus can decode code data of the enhancement layer that iscoded with a smaller amount of code by preferentially using the motionvector of the base layer over other motion vectors. This is because theinterlayer predicted motion vector has extremely highly correlation withthe motion vector in the block of the enhancement layer.

According to the present exemplary embodiment, the motion vectorintegration units 804 and 904 integrate motion vectors including thesame components in order to improve the coding efficiency of coded data.However, the motion vector integration units 804 and 904 are not limitedto this configuration. The motion vector integration units 804 and 904may select a fixed number of motion vectors. The motion vectorintegration units 804 and 904 may even be omitted.

The blocks lying in the positions illustrated in FIG. 6 are referred tofor the motion vectors of the reference motion vector group. However,the blocks and the motion vectors are not limited to this example.

According to the above-described exemplary embodiments, each of theprocessing units illustrated in FIGS. 1, 2, 3, 7, 8, 9, 12, and 14 aredescribed to be implemented by hardware. However, the processingperformed by the processing units illustrated in these drawings may beimplemented by a computer program.

FIG. 17 is a block diagram illustrating an example of hardwareconfiguration of a computer that is applicable to the image coding ordecoding apparatus according to the above-described exemplaryembodiments.

A central processing unit (CPU) 1701 controls the entire computer withuse of a computer program and data stored in a random access memory(RAM) 1702 and a read-only memory (ROM) 1703. Further, the CPU 1701executes the processing that is described to be performed by the imagecoding or decoding apparatus according to the above-described exemplaryembodiments. In other words, the CPU 1701 functions as the respectiveprocessing units illustrated in FIGS. 1, 2, 3, 7, 8, 9, 12, and 14.

The RAM 1702 includes an area for temporarily storing a computer programand data loaded from an external storage device 1706 and data acquiredfrom outside via an interface (I/F) 1707. The RAM 1702 further includesa work area for the CPU 1701 to use when executing various types ofprocessing. For example, the RAM 1702 can be allocated as a frame memoryand can provide various other areas according to need.

The ROM 1703 stores setting data and a boot program of the computer. Anoperation unit 1704 includes a keyboard and a mouse. A user of thecomputer can operate the operation unit 1704 to input variousinstructions to the CPU 1701. A display unit 1705 displays the result ofprocessing of the CPU 1701. The display unit 1705 may include, forexample, a liquid crystal display.

The external storage device 1706 is a large-capacity information storagedevice, typified by a hard disk drive. The external storage device 1706stores an operating system (OS) and the computer program for causing theCPU 1701 to implement the functions of the respective units illustratedin FIGS. 1, 2, 3, 7, 8, 9, 12, and 14. The external storage device 1706may further store pieces of image data to be processed.

The computer program and data stored in the external storage device 1706may be loaded into the RAM 1702 according to the control of the CPU 1701when needed, and processed by the CPU 1701. Networks such as a localarea network (LAN) and the Internet and other devices such as aprojection apparatus and a display apparatus may be connected to the I/F1707. The computer can acquire and transmit various information via theI/F 1707. A bus 1708 connects the above-described components.

With such a configuration, the CPU 1701 principally controls theoperations described in the above-described flowcharts.

An exemplary embodiment of the present invention may be achieved bysupplying a storage medium storing computer program code for implantingthe above-described functions to a system, and reading and executing thecomputer program code by the system. In such a case, the computerprogram code itself read from the storage medium implements thefunctions of the above-described exemplary embodiments while the storagemedium storing the computer program code constitutes the exemplaryembodiment of the present invention. An exemplary embodiment of thepresent invention also covers situations where an operating system (OS)running on the computer executes a part or all of the actual processesbased on instructions of the program code and the above-describedfunctions are implemented such the processing.

Furthermore, the above-described functions may be implemented in thefollowing mode. The computer program code read from the storage mediumis written into a memory equipped in a function expansion card insertedin the computer or a function expansion unit connected to the computer.Based on instructions of the computer program code, a CPU of thefunction expansion card or the function expansion unit executes a partor all of the actual processing to implement the above-describedfunctions.

When the exemplary embodiments of the present invention is applied tothe storage medium, the storage medium stores the computer program codecorresponding to the above-described flowcharts.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

This application claims priority from Japanese Patent Application No.2011-243936 filed Nov. 7, 2011, which is hereby incorporated byreference herein in its entirety.

1-10. (canceled)
 11. A coding apparatus capable of coding a motionvector to be used for performing hierarchical coding on a picture byusing a plurality of layers, the coding apparatus comprising: a firstobtaining unit configured to obtain a vector to be a candidate of apredicted motion vector used for prediction of a motion vector in ablock to be coded of a picture in a first layer among the plurality oflayers; and a determination unit configured to determine the predictedmotion vector in the vector to be the candidate obtained by the firstobtaining unit, wherein, if a picture in a second layer different fromthe first layer can be referred to when the block to be coded issubjected to coding, the determination unit does not regard a firstvector which is a motion vector in a block at a position correspondingto the block to be coded of a picture coded before coding of a pictureto which the block to be coded belongs in the first layer as a vector tobe the candidate, and determines any one of a second vector which is amotion vector in a block at a position corresponding to the block to becoded of a picture in the second layer and one or more third vectorswhich are motion vectors in neighboring blocks of the block to be codedof a picture in the first layer as the predicted motion vector.
 12. Thecoding apparatus according to claim 11, wherein, if a picture in thesecond layer cannot be referred to when the block to be coded issubjected to coding, the determination unit does not regard the secondvector as a vector to be the candidate and determines any one of thethird vectors and the first vector as the predicted motion vector. 13.The coding apparatus according to claim 11, wherein, if a picture in thesecond layer can be referred to when the block to be coded is subjectedto coding, the determination unit does not regard the first vector as avector to be the candidate, generates a group of vectors to be thecandidates by adding the second vector to one or more vectors which areobtained by removing a vector having the same value from the thirdvectors, and determines the predicted motion vector from the group ofvectors to be the candidates.
 14. The coding apparatus according toclaim 11, further comprising: a second obtaining unit configured toobtain a motion vector in the block to be coded; a third obtaining unitconfigured to obtain a difference between the motion vector in the blockto be coded obtained by the second obtaining unit and the predictedmotion vector determined by the determination unit; and a first codingunit configured to code the difference obtained by the third obtainingunit.
 15. The coding apparatus according to claim 11, further comprisinga second coding unit configured to code identification informationindicating the predicted motion vector determined by the determinationunit.
 16. The coding apparatus according to claim 11, wherein the firstobtaining unit obtains at least one motion vector, as the third vector,from respective motion vectors in a block adjacent to diagonally upperleft, a block adjacent to above, a block adjacent to left, and blocksadjacent to other directions of the block to be coded of a picture towhich the block to be coded belongs in the first layer.
 17. The codingapparatus according to claim 11, wherein the first obtaining unitobtains a motion vector in a block at a position corresponding to theblock to be coded of a picture to be referred to for coding the block tobe coded in the second layer as the second vector.
 18. The codingapparatus according to claim 11, wherein a picture in the second layeris a picture obtained by changing a size of a picture in the first layerat a magnification of n/m times (0<n≦m).
 19. The coding apparatusaccording to claim 11, wherein a picture in the second layer has a lowerresolution than a picture in the first layer.
 20. The coding apparatusaccording to claim 11, further comprising a second obtaining unitconfigured to obtain a motion vector in the block to be coded, wherein,if a picture in the second layer can be referred to when the block to becoded is subjected to coding, the determination unit does not regard thefirst vector as a vector to be the candidate and determines a vectorhaving the nearest value to the motion vector in the block to be codedobtained by the second obtaining unit as the predicted motion vectorfrom among the second vector and the third vectors.
 21. The codingapparatus according to claim 11, further comprising a second obtainingunit configured to obtain a motion vector in the block to be coded,wherein, if a picture in the second layer cannot be referred to when theblock to be coded is subjected to coding, the determination unit doesnot regard the second vector as a vector to be the candidate anddetermines a vector having the nearest value to the motion vector in theblock to be coded obtained by the second obtaining unit as the predictedmotion vector from among the third vectors and the first vector.
 22. Adecoding apparatus capable of decoding a motion vector from a bit streamwhich is hierarchical coded by using a plurality of layers, the decodingapparatus comprising: a first obtaining unit configured to obtain avector to be a candidate of a predicted motion vector used forgeneration of a motion vector in a block to be decoded of a picture in afirst layer among the plurality of layers; and a determination unitconfigured to determine the predicted motion vector in the vector to bethe candidate obtained by the first obtaining unit, wherein if a picturein a second layer different from the first layer can be referred to whenthe block to be decoded is subjected to decoding, the determination unitdoes not regard a first vector which is a motion vector in a block at aposition corresponding to the block to be decoded of a picture decodedbefore decoding of a picture to which the block to be decoded belongs inthe first layer as a vector to be the candidate, and determines any oneof a second vector which is a motion vector in a block at a positioncorresponding to the block to be decoded of a picture in the secondlayer and one or more third vectors which are motion vectors inneighboring blocks of the block to be decoded of a picture in the firstlayer as the predicted motion vector.
 23. The decoding apparatusaccording to claim 22, wherein, if a picture in the second layer cannotbe referred to when the block to be decoded is subjected to decoding,the determination unit does not regard the second vector as a vector tobe the candidate and determines any one of the third vectors and thefirst vector as the predicted motion vector.
 24. The decoding apparatusaccording to claim 22, wherein, if a picture in the second layer can bereferred to when the block to be decoded is subjected to decoding, thedetermination unit does not regard the first vector as a vector to bethe candidate, generates a group of vectors to be the candidates byadding the second vector to one or more vectors which are obtained byremoving a vector having the same value from the third vectors, anddetermines the predicted motion vector from the group of vectors to bethe candidates.
 25. The decoding apparatus according to claim 22,further comprising: a first decoding unit configured to decode dataindicating a difference between the predicted motion vector determinedby the determination unit and a motion vector in the block to bedecoded; and a generation unit configured to generate a motion vector inthe block to be decoded based on a prediction error decoded by the firstdecoding unit and the predicted motion vector determined by thedetermination unit.
 26. The decoding apparatus according to claim 22,further comprising: a second decoding unit configured to decodeidentification information indicating the predicted motion vector in areference motion vector group, wherein the determination unit determinesthe predicted motion vector based on the identification informationdecoded by the second decoding unit.
 27. The decoding apparatusaccording to claim 22, wherein the first obtaining unit obtains at leastone motion vector, as the third vector, from respective motion vectorsin a block adjacent to diagonally upper left, a block adjacent to above,a block adjacent to left, and blocks adjacent to other directions of theblock to be decoded of a picture to which the block to be decodedbelongs in the first layer.
 28. The decoding apparatus according toclaim 23, wherein the first obtaining unit obtains a motion vector in ablock at a position corresponding to the block to be decoded of apicture to be referred to for decoding the block to be decoded in thesecond layer as the second vector.
 29. The decoding apparatus accordingto claim 22, wherein a picture in the second layer is a picture obtainedby changing a size of a picture in the first layer at a magnification ofn/m times (0<n≦m).
 30. The decoding apparatus according to claim 22,wherein a picture in the second layer has a lower resolution than apicture in the first layer.
 31. A method for coding a motion vector tobe used for performing hierarchical coding on a picture by using aplurality of layers, the method comprising: obtaining a vector to be acandidate of a predicted motion vector used for prediction of a motionvector in a block to be coded of a picture in a first layer among theplurality of layers; and determining the predicted motion vector in theobtained vector to be the candidate, wherein, if a picture in a secondlayer different from the first layer can be referred to when the blockto be coded is subjected to coding, a first vector which is a motionvector in a block at a position corresponding to the block to be codedof a picture coded before coding of a picture to which the block to becoded belongs in the first layer is not regarded as a vector to be thecandidate, and any one of a second vector which is a motion vector in ablock at a position corresponding to the block to be coded of a picturein the second layer and one or more third vectors which are motionvectors in neighboring blocks of the block to be coded of a picture inthe first layer is determined as the predicted motion vector.
 32. Amethod for decoding a motion vector from a bit stream which ishierarchical coded by using a plurality of layers, the methodcomprising: obtaining a vector to be a candidate of a predicted motionvector used for generation of a motion vector in a block to be decodedof a picture in a first layer among the plurality of layers; anddetermining the predicted motion vector in the obtained vector to be thecandidate, wherein, if a picture in a second layer different from thefirst layer can be referred to when the block to be decoded is subjectedto decoding, a first vector which is a motion vector in a block at aposition corresponding to the block to be decoded of a picture decodedbefore decoding of a picture to which the block to be decoded belongs inthe first layer is not regarded as a vector to be the candidate and anyone of a second vector which is a motion vector in a block at a positioncorresponding to the block to be decoded of a picture in the secondlayer and one or more third vectors which are motion vectors inneighboring blocks of the block to be decoded of a picture in the firstlayer is determined as the predicted motion vector, and wherein, if apicture in the second layer cannot be referred to when the block to bedecoded is subjected to decoding, the second vector is not regarded as avector to be the candidate and any one of the third vectors and thefirst vector is determined as the predicted motion vector.
 33. Anon-transitory computer readable storage medium storing a methodaccording to claim
 31. 34. A non-transitory computer readable storagemedium storing a method according to claim 32.